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ABSTRACT 



The Even Start Family Literacy Program is intended to help 
break the cycle of poverty and illiteracy by improving the educational 
opportunities available to low- income families with limited educational 
experiences. Based on state and national evaluations, this report for policy 
makers and practitioners summarizes what has been learned about the Even 
Start family program after 10 years of demonstration and evaluation 
activities, and points out some of the directions, possibilities, and 
problems facing the program in the future. The report first describes the 
Even Start program as it has been implemented in more than 600 locations 
across the nation. The second section focuses on major changes that have 
occurred over the past decade: in the program size and administration, in the 
types of families served, in services and participation, and in program 
costs. The third section describes the kinds of effects that Even Start has 
had on participating families, with an emphasis on implications for improving 
practice. The final part of the report looks to the future and addresses 
issues of recent legislative changes, program quality, program 
institutionalization, the effect of welfare reform, and a trend toward 
improved evaluations. Contains 31 references. (HTH) 
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Even Start: Evidence from the Past and a Look to the Future 



Abstract 



The Even Start Family Literacy Program focuses on the educational needs of low-income families 
with young children. The goal of Even Start is to help break the cycle of poverty and illiteracy by 
improving the educational opportunities available to low-income families with limited educational 
experiences. To reach this goal, all Even Start projects build on existing community resources to 
integrate adult education, parenting education, and early childhood education services into a unified 
program. 

Federal law mandates an independent evaluation of Even Start projects. The U.S. Department of 
Education’s Planning and Evaluation Service (PES) has commissioned three four-year national 
evaluations since the program’s inception in 1989; two have been completed. Data from the second 
national evaluation have been reported in three interim reports, and are updated with 1 997 data in a 
final report. These volumes have been geared toward a technical or research audience. This 
volume, prepared by Abt Associates, synthesizes information from national studies and evaluations 
of the Even Start program and is geared toward a policy maker and practitioner audience. 

Implications for practice and future directions are also included. A companion final report, entitled 
National Evaluation of the Even Start Family Literacy Program: 1994-1997 Final Report, examines 
the program in more detail. 




6 



Acknowledgments 



This report presents information learned through nearly a decade of study and 
evaluation of the Even Start Family Literacy Program. Many individuals have 
contributed to the Even Start knowledge base described in this report and we would like 
to acknowledge these contributions here. 

The largest Even Start study to date is the ongoing national evaluation. This 
evaluation would not be possible without the effort of each local Even Start project 
throughout the country. Each year, in addition to the challenging work of carrying out 
the Even Start model, projects collect and report data on each of their families and 
services. Even Start State Coordinators also provide support to projects and evaluation 
contractors that is necessary for the success of the evaluation. 

The expert work group, through meetings and reviews of evaluation reports, 
provided valuable input that enhanced the quality and usefulness of the information 
gathered in the second national evaluation. It was this group that suggested producing 
a short policy volume to increase the usefulness of the large volume of Even Start 
evaluation data. The group members included: Sharon Darling, National Center for 
Family Literacy; Barbara Shay, New York State Education Department; Catherine 
Snow, Harvard University; Maris Vinovskis, University of Michigan; Mary Wagner, SRI 
International; Heather Weiss, Harvard Family Research Project; and Miriam 
Westheimer, HIPPY USA. 

The U.S. Department of Education staff provided guidance and support for the 
ongoing national evaluation and other Even Start studies referenced in this report. 
Several staff provided valuable comments and suggestions on drafts of this report. 
Special thanks are due to: Alan Ginsburg and Valena Plisko of the Planning and 
Evaluation Service; Lonna Jones of the Budget Service; James Griffin and Naomi Karp 
of the Office of Educational Research and Improvement; and Laura Chow, Patricia 
McKee, and DonnaMarie Marlow of the Office of Elementary and Secondary Education. 

Staff from several contractors have been involved in Even Start 
evaluation activities. Some of these researchers include Robert St. Pierre, Anne 
Ricciuti, Janet Swartz and Beth Gamse from Abt Associates Inc.; Fumiyo Tao, Sherry 
Khan, and Kathy McDonough from Fu Associates, Ltd; Christine Dwyer, Diane 
D'Angelo and Steven Murray from RMC Research Corporation; and Bruce Haslam and 
Elizabeth Stief from Policy Studies Associates. 



ERIC 



7 



Introduction 



This report summarizes what has been learned about the Even Start Family 
Literacy Program after 10 years of demonstration and evaluation activities, and points 
out some of the directions, possibilities, and problems facing the program in the future. 
Evidence about the nature and effectiveness of Even Start comes primarily from the 
ongoing national evaluation; speculation about the future draws on the experience of 
staff who have been involved in implementing and evaluating the program for a decade 

The report first describes the Even Start program as it has been implemented in 
more than 600 locations across the nation. The second section focuses on major 
changes that have occurred over the past decade - changes in the program size and 
administration, changes in the types of families served, changes in services and 
participation, and changes in program costs. The third section describes the kinds of 
impacts that Even Start has had on participating families, with an emphasis on 
implications for improving practice. The final part of the report looks to the future and 
addresses issues of recent legislative changes, program quality, program 
institutionalization, the effect of welfare reform, and improving evaluations. 

Many of the findings presented in this report are relevant to the performance 
objectives and indicators that the Department of Education has established in its 
strategic plan, as required in the Government Performance and Results Act (GPRA) of 
1993 (P.L. 103-62). Where this is the case, we have referenced the appropriate 
objective and indicator. The Even Start performance indicators are shown in Exhibit 4. 

We hope that this report is a helpful resource to program practitioners, policy 
makers, and researchers and that it not only captures the successes and difficulties of 
an innovative program such as Even Start, but also illustrates the long-term utility of 
having access to information from an ongoing program evaluation. 
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The Even Start Family Literacy Program 



The Even Start Family Literacy Program addresses the basic educational needs 
of parents and children up to age eight from low-income families by providing a unified 
program of (1 ) adult basic or secondary education and literacy programs for parents, (2) 
assistance for parents to effectively promote their children’s educational development, 
and (3) early childhood education for children. Projects provide some services directly, 
and build on existing community resources by collaborating with other service 
providers. 



Legislative and Program Background 

The Even Start Family Literacy Program was first authorized in 1989 as Part B of 
Chapter 1 of Title I of the Elementary and Secondary Education Act of 1965 (ESEA). 
The Even Start legislation was amended in July 1991, when Congress passed the 
National Literacy Act (P.L. 102-73), lowering the age of children served from age one to 
birth and allowing community based organizations to receive grants. In 1994, Even 
Start was reauthorized as Part B of Title I of the ESEA as amended by the Improving 
America’s Schools Act. 1 According to this legislation, the Even Start program is 
intended to: 

“...help break the cycle of poverty and illiteracy by improving the educational 
opportunities of the Nation’s low-income families by integrating early childhood 
education, adult literacy or adult basic education, and parenting education into a 
unified family literacy program. ..The program shall (1) be implemented through 
cooperative projects that build on existing community resources to create a new 
range of services; (2) promote achievement of the National Education Goals; and 
(3) assist children and adults from low-income families to achieve to challenging 
State content standards and challenging State student performance standards.” 
(P.L. 103-382, Sec. 1201). 

The 1994 legislation made the following substantive changes in Even Start: (1) 
targeting on those most in need was strengthened and services were extended to teen 
parents when they were among those most in need; (2) continuity and retention were 



1 This description of Even Start refers to the 1994 reauthorized law. Projects were not required to 
implement changes made by that law until program year 1995-96. 
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strengthened by requiring projects to serve at least a three year age range and provide 
services over the summer months; (3) the focus on family services was strengthened by 
allowing projects to involve ineligible family members in appropriate family literacy 
activities; and (4) linkages between schools and communities were improved by 
requiring stronger collaboration (partnerships) in the application and implementation 
process. Most recently, in 1996, Congress sought to further strengthen Even Start by 
passing an amendment requiring instructional services to be intensive. 

When Even Start began as a federally administered program in school year 
1989-90, grants totaling $14.8 million were awarded to 76 projects. According to the 
Even Start statute, if funding reached $50 million, the program was to be administered 
by state agencies. This level was exceeded in 1992 when the federal appropriation 
was $70 million. Most Even Start projects now are state administered, and the FY 1998 
appropriation of $124 million supports 732 Even Start projects in all states. In addition, 
family literacy programs specifically for migrant families, Indian tribes and tribal 
organizations, and outlying areas are supported through special set-aside funds (5 
percent of the total Even Start allocation) and remain under federal administration. The 
statute also authorizes discretionary grants for statewide family literacy initiatives, and a 
family literacy project in a prison that houses women and their preschool-aged children 
to be administered directly by the U.S. Department of Education (hereafter, “the 
Department”). 



Design of Even Start Projects 

Even Start’s premise is that combining. adult literacy or adult basic education, 
parenting education, and early childhood education into a unified family literacy 
program offers promise for helping to break the intergenerational cycle of poverty and 
low literacy in the nation. The Even Start program has three related goals: 

□ to help parents improve their literacy or basic educational skills; 

□ to help parents become full partners in educating their children; and 

□ to assist children in reaching their full potential as learners. 

The Basic Model. The Even Start legislation requires that all local projects 
serve families most in need of Even Start services 2 , provide three core services (adult, 
parenting, and early childhood education) and support services, provide some services 



2 To be eligible for Even Start as of 1995-96, a family needed (a) a parent who was eligible for adult 
education services under the Adult Education Act or who was within the state’s compulsory 
school attendance age range and (b) a child under 8 years of age. The definition of “most-in- 
need” is community-specific and is based on locally established criteria which must include, at 
least, family income and parent’s education level. 
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to parents and children together and provide some home-based services, integrate 
educational activities across the three core areas, coordinate service delivery with other 
local programs, conduct local evaluations, and participate in the national evaluation. 
Even Start families are required to participate in each of the three core services: 

□ Adult Education and Adult Literacy: high-quality instructional programs 3 to 
promote adult literacy, including adult basic education (ABE), adult secondary 
education (ASE), English as a second language (ESL), and preparation for 
the General Education Development (GED) certificate. 

□ Parenting Education: high-quality instructional programs to help parents to 
support the educational growth of their children. 

□ Early Childhood Education: developmental^ appropriate educational 
services for children designed to prepare them for success in regular school. 

Even Start projects also offer support services designed to facilitate the provision 
of core services. Examples of support services are transportation, child care, nutrition 
assistance, health care, meals, special care for a disabled family member, and referrals 
for mental health and counseling, services to battered women, child protective services, 
employment, and screening or treatment for chemical dependency. If possible, support 
services are to be obtained from existing providers, to avoid duplication of services. 

Even Start is intended to benefit families in several ways. While not every Even 
Start project will try to affect all of these, potential outcomes for parents include 
improved literacy behaviors (e.g., shared literacy events with children and increased 
reading and writing activities in the home), parenting behavior and skills (e.g., positive 
parent-child relationships and expectations for child), and educational and employment 
skills (e.g., improved reading and English language ability and higher education 
attainment). Goals for Even Start parents also may include growth in personal skills 
and community involvement. The potential impacts of Even Start on children include 
improved school readiness and achievement (e.g., language development and 
emergent literacy). Once children enter school, outcomes might include satisfactory 
school performance, improved school attendance, and a lower incidence of special 
education and retention in grade. 



3 In April 1996, the Even Start statute was amended to require high-quality, intensive instructional 
programs. This requirement became effective for projects in program year 1996-97. 
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Variations on the Basic Model. The legislation provides Even Start projects 
with a set of requirements that are more demanding than those of many federal 
programs, although the requirements stop short of specifying curriculum given the 
diversity of the populations served. Decisions regarding how to implement each 
requirement are left up to individual projects. For example, the legislation requires high- 
quality, intensive instructional programs; services for parents and children together; and 
services in the home. But, projects decide on the frequency and duration of program 
activities, whether the activities are primarily center-based or home-based, and whether 
to invent educational curricula from scratch or use a hybrid of existing approaches. 
Projects decide, based on the availability and quality of local services, which program 
activities will be supported by Even Start funds and which components will be 
supported by collaborating agencies. 

Most Even Start projects provide, either directly or by working with existing early 
childhood programs such as Head Start, a center-based early childhood program (Tao, 
Gamse & Tarr, 1998, p.84). Center-based programs usually incorporate elements of 
existing curricula designed for young children. Generally, school-age children through 
age seven receive Even Start services provided in conjunction with required school 
activities. Such services may take the form of homework or tutoring assistance given in 
before- and after-school child care programs and summer school activities. The extent 
to which Even Start provides early childhood services directly as opposed to delegating 
this responsibility to a collaborating agency is related to the age of the children served. 
Almost 90 percent of Even Start projects provide some or all early childhood services 
for children age four and under, 66 percent of the projects provide some or all early 
childhood services for five year olds, and 50 percent of the projects provide some or all 
early childhood services for six and seven year olds (Tao, Gamse & Tarr, 1998, p.69). 
Conversely, collaborating agencies are much more likely to provide Even Start services 
to five to seven year olds, who are of school age, than to younger children. 

Adult education services are provided in a variety of formats by different levels of 
trained personnel, ranging from volunteers to certified adult education teachers. Some 
projects offer adult education classes geared toward completing a GED, others provide 
general instruction in the basic skills of reading, writing, and math, and still others focus 
chiefly on ESL. Projects working with adults who have very low-level basic skills may 
arrange individual tutoring through organizations such as the Literacy Volunteers of 
America or provide other types of one-on-one adult education instruction during home 
visits. About 60 percent of Even Start projects provide some or all of their adult 
education services directly (Tao, Gamse & Tarr, 1998, p.69), while 40 percent of the 
projects delegate provision of all adult education services. 

Parenting education is less often available through existing agencies than are 
adult and early childhood education programs. Thus, 96 percent of the projects rely 
solely or in part on Even Start resources to deliver parenting education (Tao, Gamse & 
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Tarr, 1998, p.69). These services may take the form of group discussions, hands-on 
activities, home visits, and presentations by invited speakers. Topics addressed may 
include helping families to use learning resources, increasing parents’ understanding of 
normal child development patterns and of their role in their children’s education, and 
training parents on reading to young children. 

Educational activities often are offered in institutional settings, e.g., adult 
education classes in high schools and community colleges, and preschool programs 
associated with community-based organizations or local education agencies. In about 
17 percent of the projects, particularly those in sparsely populated rural areas, Even 
Start services are primarily home-based, with instruction tailored to each family’s needs 
(Tao, Gamse & Tarr, 1998, p. 85). 

Brief Description of Even Start Projects. To illustrate Even Start’s diversity 
and to help the reader understand what happens in Even Start projects, the next pages 
summarize the ways in which three different Even Start projects served their families 
during the 1997-98 project year. 4 



4 These descriptions are summarized from information provided by Haslam & Stief (1998) in their 
Observational Study of Even Start projects. The project names used in the descriptions are 
pseudonyms. 
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Storm Landing Even Start 

This project is administered by a school district in collaboration with a regional university, a 
local private mental health agency, and six other school districts. It operates in seven sites in 
a large geographic area including both rural and urban settings. In 1997-98, the project was 
in its sixth year of operation and served 149 families. Most sites operated adult education 
and early childhood education services three days a week for approximately six hours per 
day; staff reserved the remaining two days for collaborator meetings and home visits. 
Participants living in outlying areas were often able to attend only on days when 
transportation was provided. 

□ Early Childhood Education. Early childhood education services were provided 
for children from birth to age three, in rooms adjacent to adult education 
classrooms. Preschool children at five sites attended a variety of programs 
including Head Start and state-funded preschools. The consortium provided 
preschool services at two sites in areas with no alternative providers. Early 
childhood services for infants and toddlers were staffed by paraprofessionals, and 
children’s schedules included group activities, choice time, story time, and nap 
time. 

□ Adult Education. In all service delivery sites, the primary goal for adult 
participants was to obtain a GED. Specific elements of curriculum and instruction 
varied somewhat across sites. In one study site, adult education included small 
group, whole group, and one-on-one instruction. In another, the adult education 
classroom was individualized, with participants working at their own pace through 
workbooks. 

□ Parenting Education. A Family Services Coordinator conducted weekly, hour- 
long parenting presentations and discussions at each site. Topics included 
sessions on discipline and domestic violence, as well as nutrition, budgeting, and 
other issues related to household management and family needs. In addition, 
faculty from the regional university conducted full-day parenting seminars and 
parent activities twice a year at each site, and the project sponsored two annual 
all-site parenting seminars. The project also provided Parent and Child Together 
(PACT) time at all sites. These activities were child-centered and staff worked to 
fit them into the daily schedule. 

□ Other Activities and Services. The Family Services Coordinator provided case 
management and referral services to participants, visiting each of the seven sites 
for a full day every week or two. While on site, she met with families, conducted 
home visits, and transported families to appointments to apply for services. Adult 
education teachers also conducted home visits, often soon after a participant 
enrolled in the program and sometimes to follow up on participants who had been 
absent. 
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Lone Star Even Start 



In its fifth year of operation in 1997-98, this project was in a rural area with a population of 
13,000, collaborated with the local school district and a state college, and served 50 families. 
Most families were Hispanic; two-thirds spoke Spanish as their primary language. The 
project offered day and evening services. Fathers and dual-parent families generally 
attended during the evening, as did parents who worked in the day. 

□ Early Childhood Education. The project has its own child development center 
with a staff of seven paraprofessionals and one certified early childhood specialist. 
Two- and three-year-olds engaged in theme-based activities through the use of 
learning centers. Four-year-olds received early childhood instruction off-site in the 
local Head Start program or the state-funded public school pre-kindergarten 
program. School-age children and their siblings could participate in a literacy 
tutorial offered on site one evening a week. 

□ Adult Education. Adult goals included learning English and attaining a GED. 

Day courses, offered in the adult education center, were open only to Even Start 
adults. An adult educator, assisted by two paraprofessionals, worked with 
students in whole and small group settings, as well individually. The ESL 
curriculum included materials such as Crossroads Cafe videos, photo stories, and 
workbooks; Laubach Way to Reading skill books and audio tapes; and computer 
software. GED classes used standard adult education curriculum materials. 
Evening classes, offered by collaborating agencies, were open to the community 
and were taught by bilingual paraprofessionals. Teen parents received adult 
education at the high school. Several computers were linked to the Internet; 
software included learning packages and games. 

□ Parenting Education. This was offered two hours a week by the project director 
as well as adult and early childhood staff. Topics were developed in response to 
parents’ interests, and sessions centered around a weekly theme, incorporating 
activities, videos, and discussions drawn from parenting curricula and resources. 
Parents gave feedback on behavior management issues in a “Suggestion Circle,” 
and made theme-related products to use with their child during “Make-and-Take” 
sessions. Parents received Even Start Bucks for a variety of parenting activities 
and used these tokens to purchase toys that were available in the adult education 
classroom. Parents joined their children for PACT time in the child center twice a 
week for 30 minutes. PACT activities involved parents in an instructional role with 
their children. 

□ Other Activities and Services. Home-based instruction included an average of 
nine home visits to each family per year and focused on parenting skills related to 
child development and early literacy development. The project also provided child 
care and transportation for all classes. 
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Millersville Even Start 



In 1997-98, this project was in its fifth year of operation in a rural area with a population of 
45,000. The project served 43 families and offered services from 8:30 a.m. to 2:30 p.m., 
Monday through Thursday. Adult and early childhood education were offered daily, and 
PACT and parenting two days a week. On Fridays, project staff participated in team 
meetings, planned lessons for the upcoming week, and conducted home visits. 

□ Early Childhood Education. The project served children through three early 
childhood classes that were co-located in an “Even Start pod." A Title I preschool 
class served three- and four-year-olds, a Head Start class served four-year-olds, 
and an Exceptional Children’s class served three- and four-year-olds. All teachers 
used the state early childhood curriculum framework to guide instruction. In the 
spring of 1998, all three programs were working toward NAEYC accreditation. In 
addition to the center-based instruction, the project served school-age children 
through home visits. 

□ Adult Education. The project’s adult education component focused on 
increasing parents’ literacy skills in reading, writing, and math, and assisting 
adults to reach their own academic goals. In most cases, adults’ goals were to 
attain a GED. Daily adult education time included both independent study time 
(approximately two to two and a half hours) and small group instruction in math, 
reading, or writing (approximately one hour). Staff used the CASAS Life Skills 
Curriculum as their primary resource for small group lessons. 

□ Parenting Education. The project offered parenting education two days per 
week for an hour and a half. Staff drew on curriculum models such as Survival 
Skills for Women and 1 ,2,3,4 Parents, as well as the Department of Education’s 
Ready*Set*Read activity guides. Important contributions were made by guest 
speakers including a counselor from a local domestic violence agency, primary 
school teachers, a storyteller from the local library, personnel directors from local 
industries, and staff from local health, mental health, and cooperative extension 
agencies. PACT sessions were offered in preschool and kindergarten classes 
twice a week for 30 minutes, followed by 30 minutes of “PACT Reflection Time.” 

□ Other Activities and Services. Monthly home visits centered on an in-home, 
parent-child activity packet. Early childhood teachers usually led the home visits; 
the family specialist conducted additional visits on an as-needed basis. 
Transportation was provided for families who lived outside school boundaries and 
child care was provided for parents with children under age three. 
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The Changing Nature of Even Start 



In this section we describe some of the ways in which Even Start has changed 
over the past decade: changes in program size and administration, changes in the 
types of families served, changes in project services, and changes in program costs. 



Changes in Program Size and Administration 

The administration of Even Start changed substantially between 1989 and 1998, 
and the program significantly expanded its outreach. Exhibit 1 shows how the numbers 
of projects and families served as well as federal appropriations for Even Start have 
grown over the past decade. The first few years of the program saw a steady increase 
in the number of projects funded, as well as in the total number of families served. 

While the total number of families served by Even Start continues to grow each year as 
a function of the larger numbers of projects being funded, the average number of 
families in a project has decreased from a high of 62 in 1991 to the current average of 
54 in 1998. 

When Even Start was federally administered, programmatic guidance was 
centralized; it was provided through the Even Start Program Office, housed in the 
Department’s Office of Elementary and Secondary Education. Hence, decisions and 
responses to questions about program operations came from a single, centralized 
source. A consistent message about Even Start was heard by all grantees. When 
state agencies were given control of Even Start, in 1992, significant variability was 
introduced into program administration, for understandable reasons. For example, 

Even Start differs across states due to decisions about where to house the program 
(typically, either the state agency in charge of child development or the agency in 
charge of adult and vocational education), states’ concerns about serving their own 
areas of greatest need, states’ historical level of involvement in administering federal 
pass-through funds, and states’ depth of experience in family literacy. 

One benefit of moving Even Start’s administration from the federal to the state 
level is that local projects receive more direct and more regular technical assistance on 
programmatic issues from Even Start State Coordinators. Department staff and State 
Coordinators coordinate technical assistance issues and maintain a close relationship 
through written guidance, attendance at meetings, and electronic communication. 

Another consequence of the shift from federal to state control is increased 
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variability in the funding period for Even Start projects. Federally-administered Even 
Start projects were funded for one or two four-year grant periods. Under state 
administration, the Even Start statute continues to allow project periods up to four 
years, with individual projects being limited to a maximum of eight years of funding. 
However, some states fund projects one or two years at a time, while other states have 
maintained the federal four-year grant cycle. Regardless of the length of the project 
period, the statute requires that all projects be assessed for continuation funding on an 
annual basis. State Coordinators and local project staff have noted that uncertainty in 
future funding poses challenges to establishing healthy long-term collaborative 
relationships with school districts and other providers. 

States also vary in the amount of funding they award to local projects. When 
states were given control of funding decisions, some states decided to award a larger 
number of grants with decreased amounts of funding, thereby spreading funds across 
their states in what they perceived to be a more equitable manner. The statute requires 
states to award local projects at least $75,000 per year, although each state has the 
discretion to award one smaller grant. Data for 1998 show that the average federal 
cost per project is approximately $169,000 (Exhibit 1), and most state Even Start grants 
are between $100,000 and $200,000 per year (St.Pierre & Noonan, 1998, p.6). 

There also is variation in the level of state involvement in local evaluation. The 
Even Start legislation requires that local projects participate in a national evaluation as 
well as conduct their own local evaluation. Some State Coordinators organize local 
evaluations so that each project within the state collects the same kind of data about 
operations, participation, and outcomes. Other states allow each project complete 
autonomy to conduct their own local evaluation, and as a result, projects within a single 
state often conduct widely different kinds of studies. While no single local evaluation 
model is appropriate across all local projects, many State Coordinators refer local 
projects to the Guide to Local Evaluation (Dwyer & Frankel, 1998) and the Even Start 
Quality Indicators developed by RMC Research Corporation (Dwyer, undated). 



Changes in the Types of Families Served 

One of the Department’s objectives for the Even Start program is that 
projects will reach their target population of families that are most in need of 
services. Over the years, Even Start has widened its reach to include more diverse 
populations, and compared to earlier years, Even Start is now serving more families 
with greater evidence of disadvantage, including more teen parents and greater 
proportions of Hispanic/Latino families. Data in this section address the Even Start 
performance indicator on recruitment of most in need families. 
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Recruitment of Most in Need. Family income has been consistently low. At 
least 90 percent of Even Start families in 1996-97 had incomes at or significantly below 
the federal poverty level. The need for basic and literacy skills education for parents 
has increased since the early years of the program. Forty-five percent of 1996-97 new 
enrollees had reached at a maximum only 9th grade before enrolling; 42 percent had 
reached 10-1 2th grades but had not graduated. 

The percentage of parents under the age of 20 has grown from 9 to 1 3 percent 
in the past three years, reflecting a programmatic emphasis on serving teenage 
parents. At the same time, there has been an increase in the percentage of infants and 
toddlers in Even Start (from 27 percent in 1989-90 to 30-32 percent in 1995-97) 
reflecting a statutory change requiring service to children across at least a three-year 
age span. Among parents under age 20 who are new to Even Start, the vast majority 
(85 percent) have children who are under two years old - more than double the 
percentage for parents who are older than 20. The largest group of children continues 
to be those between three and five years of age (Tao, Gamse & Tarr, 1998, p.32-34). 

The proportion of Flispanic families in Even Start has more than doubled over the 
past 10 years, from about 17 percent of all enrollees in 1989-90 to 39 percent in 1996- 
97 (Tao, Gamse & Tarr, 1998, p.44-46). This increase is substantially greater than the 
10 to 1 1 percent change observed in the U.S. population over the same time period 
(U.S. Bureau of the Census, 1 998). Increases in the percentage of Flispanic families 
have led to increases in the proportion of non-native English speakers, whose 
participation in adult education focuses upon English-as-a-Second-Language. 

Some of the changes in the demographic characteristics of participants have 
consequences for the nature of services offered and received. Teenage parents, 
whose children tend to be younger than the children of other participants, have different 
needs both for early childhood educational services and for adult education services. 
Non-native English speakers have different needs for adult literacy education - in their 
own languages as well as in English - than native English speakers. 



Changes in Project Services and Participation 

Another objective included in the Department’s performance indicator plan 
for Even Start is the extent to which local projects provide comprehensive 
instructional and support services of high quality and intensity to all families in a 
cost-effective manner. Even Start families are required to participate in each of the 
three core service areas, except where there is a temporary absence for reasons such 
as illness or childbirth, or when a parent or child is participating under continuing 
eligibility. While the mandate for the three core instructional components has remained 
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constant since the program began, federal guidelines have become more detailed and 
explicit, to provide local projects with clearer definitions and guidance. 

The Nature of Core Instruction. In part because each project defines its own 
approach to providing instruction, there is considerable diversity in the content of the 
core services provided through Even Start. Projects borrow from and/or adapt existing 
curricular designs and materials as well as develop their own approaches. Even Start 
projects often use existing materials in their adult education and early childhood 
programs; they are more likely to develop their own materials for providing parenting 
education. This reflects the fact that adult education and early childhood education are 
more commonly offered by other collaborating agencies, while parenting education 
services, which are not widely available, generally are provided by Even Start staff. 

Who Provides Instruction. The Even Start staff who provide instruction are 
well-educated and experienced; 76 percent have at least a Bachelor’s degree, a stable 
percentage for the past four years, and over half have at least six years of post-high 
school education (Tao, Gamse & Tarr, p.66). Most projects recognize that continuing 
professional development for their own staff is critical, and as required by the statute, 
they routinely provide in-service training to their staff members. Among the most 
frequently offered topics for such training are parent and child activities, program 
planning and improvement, team building, child development, and parents’ roles as 
teachers. Only about one-quarter of projects provide in-service training to most staff on 
adult education topics, while well over half offer training on child development and 
parenting education topics. Approximately half of local projects offer training to most 
staff on support service areas, including adapting services to participants’ needs and 
working with other collaborating agencies (Tao, Gamse & Tarr, 1998, p.67). 

Service Hours Offered. The Department’s performance indicator plan for 
Even Start also includes the following indicator: Increasing percentages of 
projects will offer at least 60 hours of adult education per month, at least 20 
hours of parenting education per month, and at least 65 hours of early childhood 
education per month. The amount of instruction offered by projects is important for 
several reasons. First, the federal statute requires projects to offer intensive high- 
quality instructional services, although there is no numerical definition of intensive. 
Second, if greater amounts of services are available, then participants have greater 
flexibility in choosing when to participate. Third, some participation and outcome data 
suggest outcomes are more positive for families who participate more intensively, or for 
a longer duration (St.Pierre et al., 1995, p.253). And fourth, because so many states 
are undergoing changes in welfare requirements, and state policy changes have 
repercussions for educational participation of welfare recipients (including a substantial 
proportion of Even Start participants), Even Start projects are recognizing that 
increased amount and flexibility of instructional offerings may represent the only way to 
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maintain enrollment and participation. 5 

Recent years have seen increases in the amount of instruction offered in all 
three core areas. The amount of instruction offered in beginning and intermediate adult 
basic education, and adult secondary/GED preparation each increased by 25 to 30 
percent over the past four years, from roughly 325 hours to 415 hours per year, while 
the amount of English as a Second Language instruction increased by 12 percent, from 
300 to 335 hours (Tao, Gamse & Tarr, 1998, p.82). Over the same period, the amount 
of early childhood education offered has increased by about 10 percent, from roughly 
480 hours to 525 hours per year (Tao, Gamse & Tarr, 1998, p.83). By contrast, the 
amount of parenting education offered appears to have stabilized at about 200 hours 
per year (Tao, Gamse & Tarr, 1998, p.82). 

Over the past several years, projects have offered roughly the same ratio of 
center- to home-based hours in each core service area - home-based hours comprise 
about 1/3 of the total hours offered (Tao, Gamse & Tarr, 1998, p.87-88). The amount of 
home-based early childhood services appears to be increasing modestly, perhaps in 
response to the rising enrollment of infants and toddlers for whom there may be fewer 
existing center-based programs with an educational component. 

Participation, Retention, and Continuity. Another Even Start performance 
indicator is that projects will increasingly improve retention and continuity of 
services. The extent to which families receive the services offered by Even Start 
projects has varied over the years. The average number of hours of participation in 
adult education services rose in the early 1990s from 68 hours to 107 hours per year 
(St.Pierre et al., 1995, p.140), and has hovered around 95 hours a year for the past 
several years (Tao, Gamse & Tarr, 1998, p.107). This is equivalent to about 2 hours 
per week for a year, and is somewhat more than the national average of 80 hours for 
participation in adult basic education programs (Development Associates Inc., 1994, 
p.60). Participation in parenting education services has diminished overtime, from an 
average of 58 hours a year in 1992-93 to about 35 hours a year, less than 1 hour per 
week, for program years 1994-95 through 1996-97 (Tao, Gamse & Tarr, 1998, p.110). 
Children’s participation in early childhood educational services is calculated in terms of 
months, rather than hours, and approximately half of all children participate for more 
than 6 months (Tao, Gamse & Tarr, 1998, p.113). Finally, within any given year there is 
a clear relationship between the amount of instructional service offered and the amount 
received - when participants have more hours to choose from, they take advantage of 
this available instruction. This holds for each of the core service areas (Tao, Gamse & 
Tarr, 1998, p.117). 



5 The effect of welfare reform on Even Start is addressed at greater length later in this report. 
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About 60 percent of all families that participated in Even Start during 1994-95 
and 1995-96 remained in the program for a year or less; conversely, about 40 percent 
participated for longer than a year (T ao, Gamse & T arr, p. 1 20). This represents a 
shortening of the average period of participation since data from 1989-90 showed that 
about 50 percent of all new families remained in the program for more than a year 
(St.Pierre et al. , 1995, p.135). However, these statistics mask the fact that there is 
great project-to-project variation in when a family is considered to have officially 
enrolled in Even Start. 



Changes in Federal Program Costs 

Federal Even Start funds are used to provide core services and support services, 
to administer projects, to conduct evaluations, and for staff training. Federal Even Start 
spending grew dramatically from 1989 to 1995, was level from 1995 through 1997, and 
increased again in 1998 (Exhibit 1). Overtime, rising funding has been accompanied 
by corresponding increases in the number of projects funded and in the total number of 
families served. 

Expenditures Per Project. Average project-level federal expenditures were 
calculated by dividing the total federal funding for Even Start by the total number of 
projects funded (see the sixth column in Exhibit 1). Doing so shows that federal per- 
project expenditures grew during the first three years of Even Start, from $195,000 in 
1989 to $208,243 in 1991 . During this period the program was administered at the 
federal level and overall program expenditures were growing each year. After the 
states began administering the program, annual federal per-project expenditures 
declined steadily, from $205,882 in 1992 to $155,721 in 1997. 6 Per-project 
expenditures rose again in 1998, due to the substantial increase in program funding. 

The size of local Even Start grants vary. In the 1995-96 program year, the 
majority of projects (55 percent) had federal grants between $75,000 and $175,000, 
while 36 percent had federal grants between $175,000 and $275,000 (St.Pierre & 
Noonan, 1998, p.6). The average size of Even Start grants also varies among states 
and regions (St.Pierre & Noonan, 1998, p.6-7). In 1995-96, most states made average 



6 Even Start participants also benefit from matching funds and in-kind contributions that local 

projects are required to make, and from participating in programs to which Even Start has referred 
them. This discussion does not consider those sources of funding. The amount of the match is 
10 percent in the first year, grows to 40 percent in the fourth year of a grant, and is 50 percent in 
any subsequent year. Because Even Start now has a well-distributed mix of projects in their first, 
second, third, fourth, and subsequent years of funding, the mandated growth in the amount of 
matching funds is unlikely to account for the decline over time in average per-project federal 
expenditures. 
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grant awards between $100,000 and $200,000, but there are four states in which the 
average grant award was less than $100,000, and ten other states in which the average 
grant was greater than $200,000. States in the Northeast ($1 89,000 average) and 
Midwest ($172,000 average) tended to make larger grants than states in the South 
($155,000 average) and West ($150,000 average). 

Federal Expenditures Per Family. Exhibit 1 shows that the average federal 
expenditure for a family participating in Even Start declined quickly from a high of 
$6,024 in 1989 and has stabilized at roughly $3,000 in each of the past several years. 

In addition, the number of families served by the average project (about 54 families) 
has stabilized. 

Some Even Start projects spend relatively few federal dollars per family while 
other projects spend much more per-family. In 1995-96, almost 40 percent of the 
projects spent $1 ,000 to $3,000 in federal funds per family each year, while an 
additional 35 percent were in the $3,000 to $5,000 range. About 3 percent of all 
projects spent less than $1 ,000 in federal funds per family, and another about 1 0 
percent of all projects spent over $7,000 in federal funds per family (St.Pierre & 

Noonan, 1998, p.10). Such wide variation in federal expenditures per family reflects 
different approaches to organizing and implementing Even Start services. It also 
reflects project-to-project differences in the use of resources outside of Even Start. For 
example, some projects use locally-available adult education and early childhood 
education services, while others provide those services using their federal Even Start 
funds. 



Comparison to Head Start. On a per-family basis, federal Even Starts costs 
have declined over the past decade, from about $6,000 per family in 1989 to about 
$3,000 per family from 1993 to the present. At the same time, Head Start’s federal per- 
family costs have increased from about $2,900 per family in 1 990 to $4,600 per family 
in 1996 (ACYF, 1997). The decline, and subsequent stabilization, in Even Start’s 
federal cost per family can be attributed to a variety of factors - projects may have 
matured and become more efficient, projects may be doing a better job of using existing 
services, or perhaps projects are being pushed by states to “do more with less.” The 
increase in federal Head Start costs per family is attributable to increased program 
funding during the early 1990s, which was used primarily to improve the quality of 
existing programs and services and secondarily to serve more children. 

Comparison to Adult Education. In 1992, the federal government spent about 
$235 million on adult education programs (adult basic and secondary education, ESL 
programs). These funds were augmented by state expenditures of $810 million, so that 
total federal and state expenditures for adult education topped $1 billion (Moore & 
Stavrianos, 1995), making adult education about 10 times the size of Even Start. On a 
per-participant basis, federal basic adult education programs cost relatively little. 
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Pugsley (1990) estimated annual per-student basic adult education expenditures to be 
$160, while Development Associates (1994) estimated the average per-student cost to 
be $258, with most states spending between $100 and $500 per participant. The cost 
of JOBS training programs are estimated to range from $100 to $1 ,000 per participant 
for less comprehensive programs (Gueron & Pauly, 1991) up to $1 ,400 to $3,900 per 
participant for more comprehensive versions (Maynard, 1993; Burghardt & Gordon, 
1990). Finally, GED programs implemented by community colleges are estimated to 
cost an average of $250 per student, with ESL programs costing $154 per student 
(Hershey & Silverberg, 1993). 

Percentage of Funding Spent on Different Services. Detailed cost data were 
collected from five Even Start projects in the early 1990s. These data show that more 
than half (55 percent) of all federal Even Start funds were spent in the provision of core 
services: 31 percent for early childhood education, 15 percent for adult education, and 
9 percent for parenting education. An additional 9 percent was spent on the provision 
of support services which are designed to enable families to participate in core service 
activities. Thus, almost two-thirds (64 percent) of projects' federal funds were spent on 
the direct provision of services. Remaining federal funds were spent for program 
administration and coordination (14 percent), evaluation (10 percent), case 
management and recruiting (4 percent), and for a variety of other functions (8 percent) 
such as field trips, staff meetings, clean-up, and errands (St.Pierre et al., 1995, p.233). 7 

The finding that Even Start projects use about two-thirds of their federal funding 
for direct service provision raises the issue of whether Even Start projects are able to 
comply with the legislative requirement to build on existing services. This is a 
complicated issue for Even Start projects, and the extent to which a project uses 
existing services depends on whether those services are available locally, whether the 
services are easily accessible by program participants, whether the Even Start project 
considers the services to be of sufficiently high quality, and whether the Even Start 
project is able to make the necessary collaborative arrangements with other service 
providers. 

Given this management challenge, Even Start projects may rely more heavily on 
direct service provision early in their life cycle, switching to a greater reliance on 
existing services once they have had the time to build collaborative arrangements. The 
cost data presented above were collected in the early 1990s, when Even Start was 
new. It is possible that the percentage of Even Start funds that are spent on direct 
services as opposed to coordination of existing services may look quite different a 



7 Even Start’s two-thirds/one-third split between direct service costs and other costs is close to what 
Head Start provides. In 1991, the overall distribution of federal costs in Head Start projects was 
70 percent for direct service provision and 30 percent for other costs, with the largest categories 
being education (41 percent) and occupancy (13 percent). 
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decade later. This will be investigated as part of the third national Even Start 
evaluation. 
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What Difference Has Even Start Made to Families? 



When trying to assess the impact that Even Start has had on families, it is not 
sufficient simply to measure pre/post “gains" on test scores, employment, income, or 
parenting skills. To understand the extent to which Even Start is helpful to families, we 
need to compare gains for families who participate in Even Start with gains made by 
families who are do not take part in Even Start services, but who can, of course, obtain 
child and adult services on their own. Research has shown consistently that children 
who do not participate in any special program make gains on developmental tests, that 
some adults make gains on tests of functional literacy without being in adult education, 8 
and that low-income families have incomes which increase over time, even without 
federal intervention (e.g., St.Pierre & Layzer, 1998; Doolittle & Robling, 1994). The 
question is whether Even Start can provide a boost, change the trajectory, or alter the 
normal developmental pathway of families. Even Start seeks to do this in three ways: 

□ Increased Participation Rates. Even Start stipulates that families take part 
in three core services (early childhood education, adult education, parenting 
education). While some families not in Even Start may avail themselves of 
similar services, Even Start families are required to participate in these 
services. 

□ Integrated Services. In addition to ensuring high participation rates, Even 
Start projects provide an integrated package of services designed to build on 
each other and to have a synergistic effect, so that the effect of the whole is 
greater than the sum of the effects of each of the parts. Program developers 
often refer to Even Start as providing the "glue" necessary for families to 
engage in the complete set of services necessary to move them ahead of 
families not in the program. 

□ Increased Parenting Skills. Even Start aims to enhance child development 
not only by providing an early childhood experience, but also by increasing 
the ability of participating parents to be their child’s “first and best teacher," 
through programs of parenting education and adult basic skills education. 



8 We believe that whether adults make gains in the absence of an intervention program is likely to 
be a function of their initial level of literacy competence. Adults who cannot read are unlikely to 
learn much that is measurable by a literacy test unless they are in an intervention program that 
targets their reading problems. Adults who can read and are generally literate may well be able to 
make literacy gains on their own. 
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Questions of great interest to Congress, researchers, policy makers, and 
program practitioners are whether increased participation in the three Even Start core 
service areas, an integrated approach to providing core and support services, and 
increased parenting skills, help families more than the fragmented set of services that 
families obtain on their own. 



Hypotheses About Even Start’s Effectiveness 

A simple model of the types of effects that Even Start projects hope to produce is 
presented in Exhibit 2. Produced with the assistance of the Expert Work Group for the 
national Even Start evaluation, the model summarizes the hypotheses that underlie 
Even Start. The following text describes the model. 

Hypothesized Direct Effects. It is reasonable to assume that Even Start will 
have short-term positive effects on child development due to increased participation in 
early childhood education; on the literacy skills of adults due to increased participation 
in adult education; and on the parenting skills of adults due to increased participation in 
parenting education and parent/child together activities, as well as enhanced literacy 
skills. All of these direct effects should be apparent in a 1 -year time frame, and should 
increase in size over time. Even Start also should have direct, but longer-term, effects 
on the economic self-sufficiency of adults due to increased participation in adult 
education and mediated by subsequent enhanced literacy skills. These effects should 
occur within 2 or more years. 

Hypothesized Indirect Effects. Even Start should have longer-term positive 
effects on child development caused by continued early childhood education and 
mediated by earlier effects on parenting, enhanced literacy skills of adults, and 
enhanced economic outcomes for the family; longer-term positive effects on the 
development of nonparticipating children mediated by earlier positive effects on the 
parenting skills, literacy skills, and economic self-sufficiency of adults; and longer-term 
positive effects on parenting skills and literacy skills of nonparticipating adults mediated 
by earlier positive effects on parenting skills, literacy skills, and economic self- 
sufficiency of participating adults. The time frame for all of these effects is 2 or more 
years. 



Effect of Even Start on Program Participation 

Even Start families participate in core educational services at higher rates than 
they would have had they not been in the program. This finding affirms the hypothesis 
that Even Start is able to secure core services for families at a higher rate than they 
would have obtained for themselves. About 90 percent of Even Start families 
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participate in adult education compared with an estimated 30 to 40 percent without the 
program, about 90 percent participate in parenting education compared with an 
estimated 8 percent without the program, and about 95 percent participate in early 
childhood education 9 compared with an estimated 60 percent in the absence of Even 
Start (St.Pierre et al., 1995, p.129-131). 



Effect of Even Start on Parents, Children and Families 

Perhaps the most important objective in the Even Start performance 
indicator plan is that the literacy of participating families will improve. In almost 
all of the areas that have been measured in the Even Start national evaluation, children 
and parents in Even Start make “gains” (see Exhibit 3). The difficult question is: How 
do we know if the gains that we see are due to participation in Even Start or are larger 
than expected in the absence of Even Start? We do this by judging the gains made by 
Even Start families against the gains made by some other group, for example, the gains 
of a randomly assigned control group, the gains of families in other social programs, or 
the gains of families in norms groups. Much of the data in this section addresses 
the four end outcome indicators in the Department’s Even Start performance 
indicator plan. 

Adult Literacy Achievement. One of the outcome indicators in the Even 
Start performance indicator plan is that increasing percentages of Even Start 
adults will achieve significant learning gains on measures of math and reading 
skills. 



Each year in which the national evaluation assessed progress using measures of 
adult literacy, Even Start adults achieved statistically significant gains of 4 to 7 points 
(between .25 and .50 standard deviations 10 ) on the CASAS reading or math tests 
(St.Pierre et al., 1995, p.188-189; Tao, Gamse & Tarr, 1998, p. 152). Other adults 
achieved gains of 20 to 25 points, equal to about .25 standard deviations, on the TABE 
reading or math tests (Tao, Gamse & Tarr, 1998, p.157). However, in a small-scale 
experimental study in five projects (the In-Depth Study), families in a control group 
achieved similar gains on the CASAS, suggesting that the gains for Even Start families 
may not be due alone to participation in the program (St.Pierre et al., 1995, p. 1 85-1 87). 



9 These figures could reflect families participating under continuing eligibility. 

10 Converting gains to "standard deviation units” instead of leaving them as raw scores, puts those 
gains in a common metric so that it is possible to compare the progress of Even Start families on 
different measures. A rule of thumb for interpreting the magnitude of gains expressed in standard 
deviation units is that a gain of .25 standard deviations is considered “small,” a gain of .50 
standard deviations is "medium-sized,” and a gain of .75 standard deviations is "large” for social 
science interventions. 
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Adult Educational Attainment. Another outcome performance indicator is 
that increasing percentages of adult secondary education Even Start participants 
will obtain their high school diploma or equivalent. 

In the past decade, Even Start helped many adults attain a GED. Depending on 
the year, between 8 percent and 1 5 percent of all of the adults who entered Even Start 
without a GED or diploma attained one (St.Pierre et al., 1995, p. 1 95-1 97 ; Tao, Gamse 
&Tarr, 1998, p.162). In the In-Depth Study of a subset of five Even Start projects, 
significantly more adults in Even Start than in the control group attained a GED (22 
percent vs. 6 percent; St.Pierre et al., 1995, p.193-194). 

Based on these data we are faced with uncertainty about Even Start’s effects on 
adult literacy. Adults who participate in Even Start make gains on all of the measures 
that have been used, and gains in math appear to be larger than gains in reading. 11 
Further, the gains made by Even Start adults are comparable or larger in size than 
those observed in other studies of adult education programs (e.g., CASAS, 1992; 
Darling & Hayes, 1989). However, where data are available on adults not in Even 
Start, they too make gains, probably because they too take part in adult education 
programs. 

Even Start helps adults get a GED - a useful credential which can open doors to 
employment options and to continued education. Recent research by Murnane, Willett 
& Boudett (1 995) shows that attainment of a GED is better in an economic sense than 
not having a GED, but that it is not as beneficial as having a high school diploma. And, 
Quint, Bos & Polit (1997) concluded that there is little evidence that a GED, or a high 
school diploma for that matter, can be equated with any particular level of literacy 
performance or gains. Finally, data from the first national Even Start evaluation showed 
that no adults who entered with less than a fifth grade education were able to attain a 
GED, and only 5 percent of adults who entered having completed grades 5-8 attained a 
GED (St.Pierre et al., 1995, p.196). This presents a challenge for many Even Start 
projects because about half of adult enrollees enter with less than a tenth grade 
education and 1 5 percent enter with less than a sixth grade education. For adults with 
little or no high school experience, GED attainment remains a long-term goal. 

Children’s Language Development and Reading Readiness. The 
performance indicator for children’s outcomes is that increasing percentages of 
Even Start children will attain significant gains on measures of language 
development and reading readiness. 



1 1 This may reflect the fact that math instruction is most likely to occur in formal educational settings 
which emphasize test-related material, while opportunities to practice basic literacy reading skills 
exist outside such formal learning environments and may not contribute as much to the skills 
measured by standardized tests. 
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Both the first and second national evaluations showed that the universe of Even 
Start children learned school readiness skills such as colors, shapes, and sizes (as 
measured by a .90 standard deviation gain on the PreSchool Inventory) significantly 
faster than would be expected on the basis of normal development. After one year of 
participation, Even Start children in five projects scored significantly higher on the PSI 
than children in a randomly assigned control group. However, control group children 
caught up in the next year, when they entered preschool or kindergarten (St.Pierre et 
al, 1995, p.160-165). 

A similar pattern was found on the Peabody Picture Vocabulary Test (a test of 
receptive language). When they entered Even Start, children scored quite low on the 
PPVT, at the 9 th percentile nationally. During Even Start, children in low to moderate 
intensity programs gained a significant amount on the PPVT so that they scored at the 
19 th percentile nationally, while children who participated in higher intensity programs 
scored at the 26 lh percentile nationally. However, children in a control group gained a 
similar amount (St.Pierre et al, 1 995, p. 1 65-1 74). 

Finally, an analysis of growth rates for children who remained in Even Start for 
more than one year shows that children who remained in Even Start for longer periods 
of time may grow at a faster-than-expected rate both on the PreSchool Inventory and 
on the PreSchool Language Scale (Tao, Gamse & Tarr, 1998, p.140-146). 



It appears from all of these measures that children get a “boost” in cognitive 
development when they first are exposed to an organized school setting (either 
preschool or the public schools). Enrollment in Even Start ensures that such an 
exposure occurs at an earlier age, and so Even Start children get an earlier boost than 
control group children. The question to be addressed by future research is whether that 
early boost translates into other types of benefits for Even Start children. 

Parenting Skills. The outcome performance indicator in parenting is that 
increasing percentages of parents will show significant improvement on 
measures of parenting skills, home environment, and expectations for their 
children. 

In the second national evaluation, Even Start families gained a substantial 
amount (3.5 points, or about .50 standard deviations) on the Home Screening 
Questionnaire (HSQ), a measure of the quality of cognitive stimulation and emotional 
support provided to children by the family (Tao, Gamse & Tarr, 1998; p.147). This 
appears be larger than would be expected without Even Start, because data from the 
national evaluation of the Comprehensive Child Development Program show that low- 
income families in the control group did not make any gains on the HSQ (St.Pierre, 
Layzer, Goodson & Bernstein, 1997). 
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Several additional aspects of the home learning environment were assessed, 
and positive gains ranging from .23 to .63 standard deviations were observed in areas 
such as learning activities, story reading, books in the home, play materials, talking with 
child, and teaching child. However, in the In-Depth Study, the control group also made 
gains in each of these areas. The one area where Even Start families gained more 
than control group families was on different kinds of reading materials in the home (e.g., 
books, magazines, newspapers), something explicitly targeted by many Even Start 
projects (St.Pierre et al. , 1995, p.202-210). 

Economic Self-Sufficiency. Even Start families did not exhibit any measurable 
change on the adequacy of family resources or social support, and there was no 
measurable change in the self-efficacy of Even Start parents as assessed by self-report 
measures of sense of mastery and depression. The income of Even Start families rose 
overtime, as did the percentage of families with an employed adult, but in neither case 
were these changes greater than those seen for control group families (St.Pierre et al., 
1995, p.221-227; Tao, Gamse & Tarr, 1998, p.162-163). 

Even Start seems able to improve the home learning environment for low-income 
families. Positive changes were observed on the Home Screening Questionnaire as 
well as several other related scales. While positive gains were made on income, and 
employment, control group families made similar gains. Even Start families made few 
changes on measures of adult self-efficacy, social support, and family resources. 



Implications for Practice 

In addition to information on the overall effectiveness of Even Start, the national 
Even Start evaluations provide data on the effectiveness of selected programmatic 
practices. 

Intensity of Services Matters. A large body of research on the effectiveness of 
early childhood education programs shows that gains are enhanced by intensive 
exposure to a high-quality, center-based program (Barnett, 1995; Yoshikawa, 1995). 
Research on Even Start supports this finding in that adults and children with high levels 
of participation in Even Start's core services had larger learning gains than those with 
low levels of participation (St.Pierre et al., 1995, p. 1 80, 189-191). While this finding 
was not replicated in the second national evaluation, there were concerns about the 
quality of the outcome data in that study (Tao, Gamse & Tarr, 1998, p.129). 

Data from the first national evaluation also showed that the extent to which 
parents took part in parenting education is related to gains in children's vocabulary (as 
measured by the PPVT), over and above gains in vocabulary that result from children 
participating in early childhood education. Families who participated in a relatively low- 




31 



26 



intensity or moderate-intensity program had children who scored at the 17 th to 19 th 
percentile, while families who participated in a relatively high-intensity program had 
children who scored at the 26 th percentile (St.Pierre et al. , 1995, p.177-180). This 
finding goes to the heart of the Even Start model, showing that an intervention directed 
at parents may have an effect on their children. Unfortunately, this finding was not 
replicated using data collected from the second national evaluation, calling into 
question either the validity of the basic finding, the quality of the test data collected in 
the second evaluation (this possibility is raised by Tao, Gamse & Tarr, 1998, p. 1 29), or 
the content of the parenting education programs implemented during the mid and late 
1990s as opposed to the early 1990s. 

Literacy-Based Parenting Education is Important. If Even Start’s approach of 
training parents to be their children’s first and best teachers is to work, then projects 
need to implement a high-quality, literacy-based parenting education component. In 
Even Start’s early years, projects received strong messages from the federal level to 
focus on literacy-based parenting education. Once the responsibility for administering 
Even Start was transferred to the states, technical assistance became less focused and 
there is anecdotal evidence that parenting education has become more diffuse - a 
catch-all for a variety of parent-focused services including health education, nutrition 
education, and life skills. If this is so, it helps explain the disappearance in the second 
national evaluation of the relationship between amount of parenting education and child 
test gains. The need for a refocusing of the parenting education component of Even 
Start has prompted the Department to fund a Parenting Education Improvement 
Initiative, in conjunction with an observational study of 12 Even Start projects. The 
initiative will culminate in a “Guide to Improving Parenting Education.’’ This guide will 
be based on a conceptual framework linking different aspects of parenting education 
and children’s learning. This guide will be useful for projects in improving their 
parenting education services and for State Coordinators to use in providing technical 
assistance on how to implement literacy-based parenting education. 

Service Location Matters. Children in projects that emphasize center-based 
programs had larger learning gains than children in projects that emphasize home- 
based services. This is probably because center-based projects can more easily 
provide larger amounts of instruction. 

Project Size Does Not Seem to Matter. The number of families served and the 
grant amount of the project are unrelated to learning gains (Tao, Swartz, St.Pierre & 
Tarr, 1997, p.184). This means that children and adults do equally well in small and 
large projects. 

Parent/Child Time Matters. Families in projects that have large amounts of 
time for parents and children together had better home environments (e.g., more 
materials in the home, parent/child learning activities, approaches to discipline) than 
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families in projects that have smaller amounts of parent/child time together (Tao, 
Swartz, St.Pierre & Tarr, 1997, p. 1 84-1 85). 



Future Research Directions 



In this section we discuss the direction of Even Start research. This section 
discusses both improvements to existing research efforts, and ways in which external 
factors (e.g., welfare reform) are influencing the program, and by extension, research 
on Even Start. 



Recent Legislative Amendments to Even Start 

Congress recently enacted important amendments to the Even Start program, as 
part of the Department of Education’s appropriations act for FY 1999. This legislation 
stresses the need for local evaluations to collect data on program effectiveness, and 
requires the Department to provide technical assistance to states and Even Start 
projects to ensure that local evaluations provide accurate information on the 
effectiveness of local projects. The legislation also requires states to develop results- 
based indicators of program quality, and to use these indicators to monitor, evaluate, 
and improve Even Start programs. 



Assessing Program Quality in Even Start 

The first step toward a better understanding of the quality of Even Start 
programs was taken by the Department in the mid-1990s when it funded the 
development of a Guide to Quality for Even Start Programs (Dwyer, undated). Intended 
as a vehicle for helping program staff design their programs rather than as a set of 
performance standards, the guide specifies quality considerations for each of 10 
programmatic areas including integration of components, collaboration, recruitment, 
parenting education, home visiting, adult education, early childhood education, 
retention of families, staff development, and transitions. Research has linked some of 
these considerations to child and adult outcomes, while others are based on best 
judgments. The guide has been widely disseminated and is helpful to State 
Coordinators and project directors because it offers a guide for self-review and 
improvement. 

The national evaluation and local Even Start evaluations currently do little to 
measure the quality of program services. But quality issues are especially important for 
Even Start because projects are mandated to use local collaborative arrangements, 
when possible, as the vehicle for providing services. These services, as well as those 
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provided directly by Even Start, must be of high quality if Even Start is to produce the 
literacy outcomes expected of the program. 

Data on the quality of Even Start projects could be collected either by moving 
towards a program self-assessment of quality based on “inputs” (as Head Start 
currently does) or by defining and measuring Even Start quality in terms of participant 
outcomes. Because of the key role that collaborating agencies play in many Even Start 
projects, it would be particularly helpful to learn whether Even Start projects choose 
collaborators in a systematic way that ensures quality, or whether projects are driven by 
the requirement to build on existing services to the detriment of quality. A study that 
examines Even Start’s collaborative relationships could address questions on the 
extent to which collaborators providing instructional service share the same outcome 
goals, have a focus on literacy skills, see themselves as part of the Even Start program, 
and share data on families with Even Start core staff. 

Quality Inputs. We can define high-quality program services as those that have 
characteristics which, in previous research, have been shown to be related to positive 
program outcomes. According to this definition, we know a fair amount about what a 
high-quality early childhood education program looks like (in terms of group size, 
teacher certification, and developmentally-appropriate activities). However, we know 
much less about how to define a high-quality adult education program 12 or a high- 
quality parenting education program. We also can consider the quality of other aspects 
of Even Start, for example, administration, coordination, family advocacy, and support 
services. Two sources that may help us differentiate high versus low quality in these 
areas are the Guide to Quality in Even Start (Dwyer, undated) and the Head Start 
Performance Standards (Head Start Bureau, 1992). 13 

Quality Outputs. Another way of assessing program quality is to measure 
program outputs. By this definition, a program that produces good program outputs is a 
high-quality program. Congress recently enacted legislation requiring results-based 
educational performance measures for children participating in Head Start programs. 
Similarly, Even Start has established preliminary output standards through the 
Department of Education’s leadership in complying with GPRA and in its performance 
indicator plans (U.S. Department of Education, 1997). And, the recent Reading 
Excellence Act specifies that states must develop indicators of quality outputs, both for 



12 The Department’s current study of effective adult education practices for first-level learners is 
examining this issue. 

1 3 Compliance with the Head Start performance standards is a requirement for receipt of federal 
Head Start funding. The standards specify the types of educational services, operations, and 
facilities that are required; the health, medical, and nutritional services that must be provided; and 
the social services and parent involvement that must be offered by Head Start programs. 
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adults and children, for use in monitoring, evaluating, and improving programs. 

Given this interest in measuring quality outputs, it would be possible to set 
performance goals for individual Even Start projects as well as for the program as a 
whole. Such performance goals might, for example, take the following form: 

□ 50 percent of all Even Start children who are entering kindergarten will score 
at or above the 50 th percentile on PPVT. This percentage will increase by 2 
percentage points per year over each of the next 10 years. An alternative 
example: Even Start children will achieve an average annual gain of at least 
.50 standard deviations (7 items) on the PPVT. 

□ After one year in Even Start, 50 percent of all adults who are in ABE will 
score at or above the 50 th percentile on either the TABE or CASAS reading 
test. This percentage will increase by 2 percentage points per year over each 
of the next 10 years. An alternative example: Even Start adults will achieve 
an average annual gain of at least .50 standard deviations on the TABE or 
the CASAS reading test. 

Setting standards assumes that there will be a measurement system for 
assessing whether those standards are met. In the ESPIRS, Even Start already has 
the basis for such an assessment system, and the Department’s Even Start 
performance indicator plan (for use in complying with GPRA) represents a first step in 
setting output targets, if not standards for the national program. 



Studying the Institutionalization of Even Start 

There are at least two ways in which Even Start, and family literacy more 
generally, can be seen as becoming institutionalized: (1) the statute limits Even Start 
projects to a maximum of eight years of federal funding, but projects can obtain other 
funding to continue operations; (2) the influence of Even Start can be broadened at the 
federal and state levels by including family literacy services as components of other 
federal programs or by providing state funding for Even Start-type programs. 

Continued Local Funding. After a maximum of eight years of federal funding, 
an Even Start project no longer is eligible to receive federal Even Start funds. While not 
a central or explicit goal of the program, a reasonable indicator of Even Start’s success 
is the extent to which projects continue providing family literacy services after their 
federal grant has expired. The issue of project self-sufficiency is being studied by Ann 
Martinez at Texas A&M University who has sent out surveys to each of the original 
1989 cohort of projects to find out what has happened after they completed their eight 
year federal funding cycle. This study will examine whether these projects were able to 
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continue family literacy services after Even Start, and if so, how they did it. 

Expanded Federal/State Funding for Family Literacy. Another possible 
outcome for Even Start is the generalized institutionalization of family literacy programs 
in communities. There is substantial evidence that this is happening at the federal 
level. For example, recent legislation including the Reading Excellence Act, the Adult 
Education and Family Literacy Act, and the Head Start legislation, greatly broadened 
the scope of federal programs that include family literacy services. 

There is a similar proliferation of family literacy services and programs at the 
state-level. Several states, including Kentucky, Arizona, South Carolina, and New 
Mexico, have introduced or passed their own family literacy legislation. In 1989, South 
Carolina provided $2.3 million in funds for parent education and family literacy pilots in 
21 school districts. In 1993, legislation required that all South Carolina districts 
implement similar programs based on the experience of the original districts. Kentucky 
has a state-funded family literacy program, the Parent and Child Education program. 

These federal and state efforts are important evidence of the increasing appeal 
of family literacy and could prove important to the self-sufficiency of Even Start projects 
after the conclusion of federal Even Start funding. 



Studying the Effect of Welfare Reform on Even Start 14 

Each year about 40 to 50 percent of the families in Even Start rely on 
government assistance for their primary source of income. Thus, changes in the 
federal welfare system have important implications for Even Start participants. In 
particular, the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 
(PRWORA) brought significant changes in time limits and work requirements for welfare 
recipients. The federal law now limits lifetime welfare benefits to five years, with the 
provision that 20 percent of adult recipients can be exempted as hardship cases. The 
law also requires an increasing number of adults to work each year, with half of the 
recipients having to work at least 30 hours a week by 2002. Through the waiver 
system, states have been allowed to set more rigid limits in terms of the benefits that 
are allowable and the time frame in which adults and teens can receive these benefits. 

The implementation of the Temporary Assistance for Needy Families (TANF) 
block grant to states under the PRWORA has affected Even Start projects in a number 



14 The information in this section is drawn from Alamprese & Voight (1998) who conducted five case 
studies of adaptations that family literacy projects have made to meet the new requirements faced 
by welfare reform clients. This work was done for the National Center for Family Literacy and the 
Knight Foundation. 
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of ways. Program staff are adapting their adult education, early childhood education, 
parenting, and parent-child interaction services to meet the needs of welfare recipients. 

Adult Education. The major assumption that has guided the delivery of adult 
education services in Even Start is that basic skills education is critical in enabling 
adults to carry out their responsibilities as economic providers, parents, and teachers of 
their children. Welfare reform has shifted the types of Even Start services for adults 
away from non-work-related adult basic education activities and towards job preparation 
and job search skills taught in a literacy context and support services that focus on job 
placement services. Depending on the state’s laws, Even Start staff work with local 
welfare agency case managers to design services that qualify under the welfare reform 
law and that include a basic skills component (usually GED preparation or ESL 
instruction). Often, the time allotted to basic skills has to be reduced to allow for the 
inclusion of work-readiness activities taught in a literacy context. 

One strategy that some Even Start staff are using to address the reduction in 
time for direct basic skills instruction is to teach basic skills in the context of job 
preparation skills. For example, adults are taught reading and writing in the context of 
preparing a resume and completing job application forms and letters. Participants also 
are taught the higher order skills of problem-solving and decision-making in their work 
preparation experiences. In job shadowing, participants observe a variety of jobs, 
analyze the skills required for these jobs, and then compare the required skills to those 
that they possess or are working to improve. 

To meet the welfare reform work experience requirement mandated in many 
states, Even Start staff in projects where services are in one location, such as an 
elementary school, are working with building personnel to arrange for work experience 
placements in the cafeteria, housekeeping, or the office. In this type of arrangement, 
the participant is able to meet the welfare reform work experience requirement while 
remaining in the Even Start program setting. In states where recipients are limited in 
the time in which they can prepare for work, Even Start staff often offer basic skills 
services after the recipients’ work day. This type of extended learning requires that 
family literacy participants commit to attending activities to enhance their basic skills 
after a full work day, which is difficult in terms of participants’ energy and time for 
carrying out family responsibilities. 

New welfare requirements mean that Even Start staff serving welfare recipients 
must develop new ways of teaching basic skills in the contexts of job preparation and 
job placement activities as well as offering extended learning experiences for working 
participants. These requirements also have led to new outcomes for which the program 
may be held accountable. While achievement of gains in basic skills has been the 
traditional outcome for the adult education component of Even Start, job placement and 
job retention now are being required for welfare recipients participating in family literacy 
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programs. This change in program goals has prompted Even Start staff to be 
concerned not only about the participants’ learning but also about the status of these 
individuals with regard to work readiness and work attainment. 

Early Childhood Education. Even Start projects serving welfare recipients 
have changed the amount and the schedule of time that early childhood education is 
available for recipients’ children as well as the content of early childhood instruction. In 
programs where the schedule has been extended to accommodate adults’ participation 
in work experience as well as classes, the time for early childhood education also has 
been lengthened. In some instances, this has changed the demand for support 
services such as transportation, where additional services are needed. Projects that 
offer extended basic instruction for working participants are providing activities for 
children while their parent is receiving instruction in basic skills or ESL. 

One way that Even Start projects integrate content between early childhood 
education and adult education is by incorporating work themes in early childhood 
instruction. For example, an early childhood education instructor may address the 
same occupational content that is being taught in adult education by using children’s 
books about these occupations and having the children dress in costumes representing 
the roles of workers in the occupations. The intent is to socialize children to the world 
of work and to provide content that can be used in discussions between parents and 
children. This type of integration requires close coordination between instructors and 
the resources to purchase additional materials for the early childhood instruction. 

Parenting Education and Parent/Child Time. With the change in schedule of 
adult education and the requirement for work, some Even Start projects have had to 
reduce the frequency with which parenting education is offered. As Even Start adults 
adjust to the demands of welfare reform, demands which place new time stresses on 
them, project staff often reframe parenting instruction to include issues that extend 
beyond the content usually addressed in parenting education. For example, some 
Even Start projects include time and fiscal management as part of parenting education. 

All Even Start projects are required to provide some services to parents and 
children in a joint setting, often with the intention of improving the child’s cognitive skills. 
The time changes necessitated by welfare reform often result in a reduction in the 
amount of parent and child time or in a change in the way this component is delivered 
in Even Start projects. Some projects now provide parents with packets of materials 
that they can use with the child in carrying out exercises at home. Parents are 
instructed how to use the materials prior to taking them home and then discuss their 
activities afterwards. In projects with a substantial home visit component, the home 
visitor may observe the parent-child interaction in the home and provide guidance to the 
parent in these activities. 
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Implications for Evaluation and Program Effectiveness. The adaptations 
that Even Start projects are making to address welfare reform have implications for 
evaluation and program effectiveness. The national evaluation’s ESPIRS system 
needs to reflect the new types of activities that are being offered in each core service 
area, particularly in adult education concerning the provision of extended basic skills 
instruction and career awareness, job search, job preparation, job placement, and job 
retention/follow-up activities. Since the time in which activities are offered has changed, 
an accurate accounting of the time that participants spend in each core service area 
continues to be important. In addition to changes in program components, Even Start 
projects have extended their partnerships and collaborations to include new 
organizations and agencies, and these new arrangements need to be documented. 

The types of outcomes for Even Start participants and the measurement of these 
outcomes may well change as a result of welfare reform. 

Perhaps most important, the changes described above including a reduction in 
the amount of time for adult basic skills instruction, a refocusing of parenting education 
away from literacy and towards life skills, and a reduction in the amount of parent/child 
time offered by projects, run counter to the theory underlying Even Start. Some of the 
positive findings from past evaluations, summarized earlier in this report, were that 
larger amounts of parenting education were associated with higher PPVT scores for 
children, and that larger amounts of parent/child time were associated with better home 
literacy environments. If complying with welfare reform means that Even Start has to 
back away from its emphasis on parenting and parent/child activities, the program could 
become less effective in some areas. 

Alternatively, other outcomes such as increased levels and rates of employment 
and reduced welfare dependency may assume a more important position in Even Start. 
The expectations for participants and the measures used in documenting outcomes 
need to be reviewed to determine their appropriateness in light of program changes. 



Improving Even Start Evaluations 

Since Even Start’s first year, legislation has included evaluation requirements at 
both the local and national levels. These two levels of evaluation respond to differing 
information needs of the Department and local Even Start projects. Though the 
legislative mandate has changed slightly over the years, the national evaluation’s basic 
purposes have remained the same - to describe Even Start projects and participants, 
to examine the performance and effectiveness of Even Start projects, and to identify 
effective Even Start projects for use in program improvement and technical assistance. 
Two cycles of four-year national studies have been completed, and a third national 
evaluation is underway. There is substantial continuity across the three national 
evaluations, but each had its own special focus and challenges. 
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First National Evaluation (1989-90 through 1992-93). The first national 
evaluation was broad in scope, addressing questions such as: “What are the 
characteristics of Even Start participants? How are Even Start projects implemented 
and what services do they provide? What Even Start services are received by 
participating families? and What are the effects of Even Start on participating families?" 
To answer these questions the evaluation developed the National Evaluation 
Information System (NEIS) which used paper and pencil forms that had to be key- 
entered as well as optically-scannable forms to collect data on participant 
characteristics, project implementation, and participant outcomes from all projects. In 
addition, the evaluation included an experimental component (the In-Depth Study) in 
which families in five sites were randomly assigned to be in Even Start or a control 
group, and were measured three times over an 18-month period. 

The study provided useful information about Even Start’s early implementation 
but it did not provide solid answers to questions about Even Start’s effectiveness. Data 
on literacy outcomes were collected on children and adults in all projects. These 
national data showed consistent gains overtime, but without a control group there was 
no way to know whether the gains resulted from participation in Even Start. The In- 
Depth Study provided experimental evidence about Even Start’s effectiveness, but it 
suffered from a small sample size (only five projects were able to commit to the 
experimental design), sample attrition from that small original sample, and a lack of 
information on the control group’s experiences. 

The first national evaluation did describe whether early Even Start projects were 
able to implement the program as intended. Careful documentation of program 
implementation helped the Department and grantees agree on the definition of key 
program terms by answering questions such as “What counts as adult education in 
Even Start?” and “Who counts as a program participant?” The Department used data 
from the first evaluation to identify areas where Even Start projects needed technical 
assistance; in particular, improving the literacy focus and intensity of their parenting 
education components, engaging adults in adult education, and recruiting and retaining 
families. 

Finally, information from the first national evaluation was used to improve the 
program through legislative changes. Findings that showed a fairly low year-to-year 
retention rate were used to modify the legislation to require year-round services so that 
families would remain involved in Even Start throughout the summer and into the next 
year. Other substantive changes made to the legislation that were informed by the 
study included focusing program targeting on those most in need, requiring that 
projects serve at least a three-year age range of children, allowing projects to serve 
teen parents, and allowing the involvement of ineligible family members in appropriate 
family literacy activities. The finding that there was a relationship between the amount 
of participation in Even Start and child/adult test gains, coupled with similar early 
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findings from the second evaluation, provided evidence that resulted in an amendment 
in 1996 requiring Even Start services to be intensive. 

Second National Evaluation (1993-94 through 1996-97). In the Department s 
second national evaluation the data collection instrument was improved, converted to a 
PC-based system, and renamed the Even Start Information System (ESIS). The 
administration of literacy tests was restricted to children and adults from a 10 percent 
sample of projects (approximately 60 out of 600), called the Sample Study, although 
program and participation information was collected from all projects. No control group 
component was included in the second evaluation. 

The Sample Study was intended to provide a national snapshot of Even Start’s 
outcomes over a four year period, and to relate Even Start practices to outcomes for 
use in designing technical assistance to projects. However, small within-project sample 
sizes, project and family attrition from the sample, and questionable quality of test data 
collected by local project staff left the Sample Study unable to provide information on 
the outcomes of individual Even Start projects or on effective practices, although it was 
possible to aggregate data and examine Even Start’s outcomes across all projects in 
the Sample Study. Further, without a control group, the Sample Study could not 
address lingering questions about Even Start’s impact. 

While the second national evaluation had flaws, it provided the Department with 
useful information to improve the program nationally. Implementation data collected 
through the ESIS allowed the Department to track changes in the population served 
over an eight-year span, and the Sample Study provided national-level data on the size 
of gains made by Even Start participants. Early ESIS data provided evidence 
corroborating the positive relationship between service intensity and family outcomes 
found in the first national evaluation. 

Perhaps the most important innovation in the second national evaluation was a 
computer program that helps grantees use national data at the local level by generating 
summaries of their own ESIS data. Another innovation was the development of annual 
profile reports for each Even Start project that could be used for continuous program 
improvement. First sent to each project and State Coordinator in the fall of 1 997, these 
reports compare each project’s data on several important variables to state and 
national data, and to other projects with similar characteristics. 

Third National Evaluation (1997-98 through 2000-01). The Department 
funded the third national Even Start evaluation in the fall of 1997. The data collection 
instrument was again updated and renamed the Even Start Performance Information 
Reporting System (ESPIRS). Major improvements to the reporting system include the 
addition of a section asking parents to report the types of literacy-related activities and 
behaviors in which they and their children engage as well as the kinds of literacy-related 
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tasks that their children can perform, and an updating of the project profile system 
developed in the second national evaluation. 

To respond to criticisms that the first and second national evaluations had not 
adequately addressed the issue of Even Start’s effectiveness, the third national 
evaluation provides for an experimental study to test the effectiveness of Even Start in 
20 fully-implemented projects with a total of 400 Even Start families and 200 control 
group families. 

Each of the national evaluations has shown that Even Start is a complicated 
program that takes time to understand and implement fully. The focus on literacy for 
the family as a unit is a special challenge, as is the requirement to build on existing 
services to create a unified family literacy program in a community. Maintaining a 
literacy focus and ensuring that collaborators share the goals and objectives of Even 
Start are critical, since Even Start projects are held accountable for literacy outcomes, 
regardless of collaborators’ objectives. By integrating the three core instructional 
components, Even Start projects seek to provide a value-added dimension to families’ 
literacy experiences and outcomes. These characteristics make Even Start unique and 
exciting, but also difficult to evaluate. 

Working Toward Wider Use of Evaluation Data. The first two national 
evaluations were successful at describing Even Start participants over time. For 
example, the evaluation identified a dramatic rise over time in the percentage of non 
English-speaking families who participate in Even Start. How to best serve this growing 
population is one of the most important future issues for Even Start. What the national 
evaluation needs to do a better job of in the future is capturing the results of Even Start 
and ensuring that the information obtained is useful for many different stakeholders. 

The key to collecting useful information is to focus on data that will improve the 
program. Given the need for outcome information in order to identify effective practices 
and the requirement of GPRA to report on program performance indicators for Even 
Start (as well as for all other Departmental programs), continuous improvement that is 
based on rigorous and objective assessment will be important to the future of Even 
Start. The Department will continue to examine ways to strengthen the outcome 
portion of the national evaluation, with one option being to return to the practice used in 
the first national evaluation, in which outcome information was collected on all families. 

Working Toward High Quality Data in the National Evaluation. The national 
evaluation has been made possible by a partnership between the Department, national 
evaluation contractors, and local projects. Except for the first evaluation’s In-Depth 
Study and the planned Experimental Study in the third evaluation, the national 
evaluation has relied on local projects for data collection. This means that data quality 
has varied across projects and across evaluation years, at times to the detriment of 
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data usefulness at any level. For example, outcome data from the two most recent 
program years (1995-96 and 1996-97) no longer show a positive link between the 
amount of participation and child or adult literacy outcomes. It is possible that this 
results from a lack of consistency across projects in the administration of child and adult 
literacy tests. 

As the Even Start program grows larger (from fewer than 100 grantees in 1989 
to more than 700 in 1998), data quality and the resources it takes to train all Even Start 
grantees to collect data grow in importance. While it would be possible to estimate 
national statistics based on a sample of projects, there are several good reasons to 
include all Even Start projects in the national evaluation. First, the more projects 
involved, the better able the evaluation will be to identify promising practices, through 
correlational analyses and case studies. In addition, the national evaluation is currently 
.the source for all tracking information on projects. Without a universe data collection, 
the Department would have no comprehensive list of all Even Start grantees or their 
characteristics. Finally, having all local projects collect a common set of data means 
that analyses can be done at the national, state, and local levels. 

Balancing National and Local Evaluation Needs. Another challenge in 
evaluating Even Start is balancing the data needs of national and local evaluations. In 
response to legislative mandate, the national evaluation is meant to track trends over 
time, provide performance information, identify promising practices, provide a national 
and state comparison for local projects, help the national program office identify 
technical assistance priorities, and improve Even Start nationally. At the same time, 
local projects are required to conduct a local evaluation to help improve services and 
demonstrate positive results to appropriate stakeholders. 

While recognizing that the national evaluation is evolving into a more useful tool 
for local projects, project staff worry that they are asked to spend time and energy 
collecting and submitting data for the national evaluation, but that those data are of 
limited use at the local level. Project staff have commented that the national evaluation 
could do a better job of capturing what goes on in individual Even Start projects, 
thereby illustrating the diversity of projects across the country. Unfortunately, the 
characteristics of the universal data collection instrument which make it useful for 
informing Congress about the national Even Start program also make it less useful to 
individual projects. The Department has stressed that the national evaluation must first 
respond to national information needs, which means, by definition, that it cannot 
provide data which are maximally useful to local projects. Though the national 
evaluation has been limited in its ability to serve local needs, progress is being made. 
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The Department, through the national evaluation and other activities, has tried to: 

□ Help Even Start projects understand evaluation and its uses. Sessions 
at national conferences have focused on evaluation issues, e.g., the ways in 
which evaluations are used by Congress and the Department, the different 
roles played by members of evaluation teams, the characteristics of good 
evaluations, the importance of collecting unbiased data, the usefulness of 
evaluation designs that allow strong inferences, the rationale underlying 
decisions that are made for the national evaluation, and explanations of the 
kinds of measurements used in the national study. 

□ Improve the quality of local evaluations. Sessions at national conferences 
and publications issued by the Department have focused on how to conduct 
useful local evaluations (Dwyer, 1998), on networking among local 
evaluators, and on the different types of studies conducted at the local level 
and the uses made of those studies. The Department has conducted a 
recent review of more than 100 local evaluations with an eye toward 
understanding those studies and providing guidance for improvement 
(St.Pierre, Ricciuti & Creps, 1998). 

□ Analyze data with an eye to findings that have programmatic 
implications. National evaluation reports have tried to include analyses that 
provide local projects with data-based information about effective program 
practices. Examples include analyses of the relative effectiveness of varying 
levels of service intensity, different types of instructional staff, and varying 
amounts of support services. 

□ Provide each project with access to its data. Each Even Start project 
collects descriptive data on program operations and family characteristics, as 
well as data on parent and child outcomes. These data are collected by local 
project staff using a nationally-developed data base system and are sent 
annually to the national evaluator for aggregation and analysis. The data 
base allows local project staff to access the data that they input, to produce 
descriptive statistics for local use, or to output the data for analysis using any 
statistical package. Many local projects are interested in this feature, but so 
far, few have staff with the technical skills to take advantage of it. 

□ Provide project-level feedback reports. The national evaluators have 
developed a project profile reporting system where each year, the data 
supplied by local projects are analyzed to form national and state aggregates, 
as well as aggregates for several different types of projects (e.g., by size of 
project, age of project, need level of families served). A profile report is then 
produced, comparing the values of 20+ descriptive and outcome variables for 
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each project (e.g., average of hours of instruction offered and received, 
program participation rates, family income and employment levels, 

• percentage of children and adults who engage in different literacy activities) 
against national and state averages, as well as averages of similar projects. 

The State Role in Evaluation. States administer the Even Start program and 
need some sort of accountability tool. Yet at present, there are no standards for states 
to use in deciding whether their Even Start projects are making sufficient progress. 
Given this information gap, it is not surprising that several states have assumed 
responsibility for planning and conducting studies of their local projects, either by using 
administrative funds or by coordinating local project evaluations. 

The existing state role in evaluation seems poised for change because the 
recently-enacted Reading Excellence Act amended the Even Start law by requiring 
states to develop results-based indicators of program quality to be used in monitoring 
the performance of local projects. States now have a unique opportunity to set results- 
based quality standards, and having state standards could improve evaluation at the 
local level and increase accountability. At the same time, the new prominence of states 
in Even Start evaluation could lead to overlap among national, state, and local 
evaluation activities. Careful coordination among these different levels of evaluation will 
be needed to avoid overburdening local projects. 

Complementary Evaluation Studies. Because the national evaluation cannot 
answer all of the research questions that are asked about Even Start, the Department 
has funded additional studies of Even Start, each with a special focus. For example, 
the Observational Study of Even Start Projects is studying 12 projects that are at least 
in their third year, that have evidence of being fully implemented, and that have 
produced positive outcomes for at least two years. The study is examining outcomes 
for families, quality of implementation, and linkages across Even Start’s three 
components. Through this study, the Department hopes to facilitate and assess 
approaches to continuous program improvement in Even Start projects based on clear 
outcome goals for children and families, program quality standards, rigorous and 
objective assessment of program results, and the use of evaluation results to monitor 
progress and enhance program quality. As described earlier, the second part of this 
study will help projects improve their parenting education component. Information 
gathered from working with these projects over three years will help the Department 
provide assistance to all Even Start projects on local evaluation and parenting 
education. 

Other complementary studies being funded by the Department include a 
synthesis of state and local Even Start evaluations, a synthesis of research on family 
literacy, a study of the continuity of services between Even Start and Title I, and an 
analysis of the population eligible for Even Start. All of these should provide useful 
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information in helping to frame the future Even Start research agenda. 



Future Evaluation Options 

Even Start has an ongoing national evaluation for all grantees that informs 
Congress and the Department. Future options for evaluation ought to focus on issues 
that will help local projects improve their services, as well as on the overall 
effectiveness of Even Start. 

Focus on Literacy Outcomes for All Projects. Data on the literacy skills of 
Even Start participants is important at all levels. Locally, Even Start projects need to 
assess literacy progress to know what, if any, changes to make in their services. Data 
on literacy outcomes also are important for Even Start State Coordinators, who now 
must specify quality standards and decide which projects are making sufficient progress 
to receive continued funding. Nationally, Congress holds Even Start accountable for 
progress toward outcome achievement targets set in the national performance indicator 
plan, required by GPRA. Since the national evaluation collects information on all 
projects, and all projects would benefit from having outcome data, one option is for the 
national evaluation to designate a common set of rigorous and objective outcome 
measures to be used in all projects, and also give local projects flexibility by allowing 
them to choose from a set of approved measures. 

Strengthen Local Evaluation for Continuous Improvement. How can local 
projects best be improved? A recent analysis of local Even Start evaluations (St.Pierre, 
Ricciuti & Creps, 1998) shows that local projects do not consistently produce high- 
quality, useful, local studies and calls into question the utility of local evaluations, as 
currently structured, for improving projects. Like many other aspects of Even Start 
projects, the content and quality of local evaluations varies. Variation in state and 
Department guidance, project maturity, available funds, and project needs contributes, 
not surprisingly, to variation in local evaluation designs. While the synthesis found 
some local evaluations that were well done and useful to projects, others were of low 
quality and of little use to a local project. 

Program improvement efforts have been haphazard, occurring mostly through 
trial-and-error and through the hard-learned experiences of program staff. This is one 
way of improving projects, but greater gains could be made by accompanying these 
personal, anecdotal methods with a data-based, systematic assessment of program 
strengths and weaknesses. Through dissemination of the Guide to Quality in Even 
Start (Dwyer, undated) and through evaluation training sessions, the Department has 
been working with local projects to modify their approach to evaluation by engaging in 
such a continuous improvement effort. Relevant systematic continuous improvement 
efforts have been described by Haslam & Stief (1998) in an observational study of 12 
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Even Start projects, by Alamprese (1996) in studies of workplace literacy programs, and 
by Appel (1998) in her work with local Even Start evaluations. 

Examine the Quality of Even Start Services. Although the Department has 
issued guidelines for establishing a high-quality Even Start project, there has been no 
systematic assessment of the quality of Even Start projects or of individual Even Start 
services. Options for assessing program quality include a self-assessment of the extent 
to which each project’s services match the Even Start quality guidelines (the Guide to 
Quality has an accompanying program self-assessment tool) and a comparison of the 
gains made by Even Start families against a set of performance standards. Finally, 
collaborating agencies play a key role in many Even Start projects, and it would be 
particularly helpful to do a study of the collaborations that Even Start projects arrange. 
Such a study could examine whether Even Start projects choose collaborators in a 
systematic way that ensures quality, or whether projects are driven by the requirement 
to build on existing services to the detriment of quality. 

Continue to Study Short- and Long-Term Effects. The basic question about 
Even Start’s effectiveness has not been answered to everyone’s satisfaction. The first 
national evaluation provided useful data on learning gains from all projects, as well as 
data from five projects which implemented high-quality randomized experimental 
evaluations. However, no group of five projects can represent Even Start very well, and 
the findings from this study, while important, cannot be generalized to Even Start as a 
whole. The second national evaluation provided data on learning gains from a random 
sample of all projects, but with no control group it is difficult to attribute gains to Even 
Start. The third national evaluation plans to provide better answers about Even Start’s 
effectiveness by implementing randomized studies in 20 projects across the nation, and 
by collecting pretest/posttest data on child and parent behaviors from all Even Start 
families in all projects. 

Through the studies done to date we have learned that Even Start adults and 
children make short-term literacy gains and that many adults acquire a GED, even 
though the average length of participation is less than a year. While the effect on 
acquisition of a GED seems clearly attributable to Even Start, we are unsure of the 
extent to which the literacy gains are due to Even Start or to normal development. This 
question will be addressed by the experimental study currently being conducted as part 
of the third national evaluation. If early findings are favorable, the experimental study 
could be extended to determine whether the short-term effects of one year of Even 
Start participation lead to the hypothesized longer-term effects that are illustrated in 
Exhibit 2. 

Unfortunately, the severe need level of most Even Start families combined with 
the fact that most families leave the program within their first year of participation, may 
truncate the effectiveness of the Even Start model. Though it could not be done in an 
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experimental'fashion, it would be useful to follow families who participate in Even Start 
for an extended period of time, in an attempt to determine the long-term value of 
persistent participation. 

States could help with long-term studies of Even Start. The mobility of Even 
Start families after they leave the program (and in many cases during their Even Start 
participation) complicates any national long-term follow up. However, some states are 
in the process of building detailed student data bases for tracking students, data’ bases 
which could be useful both at the state level and nationally to provide information on 
students who were once in Even Start. For example, the Florida State Department of 
Education has put into place a data base of student information that details student- 
level information each year. To examine the long-term effects of Even Start, it would be 
useful if data bases like these contained an element that would assign a code for each 
student indicating participation in Even Start. Ideally, state-level data bases would 
indicate any participation (current or former) in Even Start for children in all grades. 

The diversity of the current set of Even Start studies, including the national 
evaluation, descriptive case studies, and syntheses of research and evaluations is 
already broadening the types of information available about Even Start implementation 
and outcomes. The Department has made important improvements to the third national 
evaluation, including the use of a control group for isolating Even Start’s impact and a 
new set of family progress indicators. Upcoming improvements to the reporting system 
will focus on accurate measurement of important factors such as service intensity and 
participation. The control group study could also be a vehicle for examining some of 
Even Start’s cost and quality issues. The current portfolio of Even Start studies, in 
conjunction with exploration of the options discussed, represents the future of Even 
Start’s evaluation efforts. 
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Exhibit 1 

Even Start Size and Expenditures, by Fiscal Year 



Fiscal 

Year 


N of 

Projects 

[A] 


N of 

Families 

[B] 


N of 

Families 

Per 

Project 

[B/A] 


Federal 
Even Start 
Expenditure 
[C] 


Federal 
Even Start 
Expenditure 
Per Project 
[C/A] 


Federal 
Even Start 
Expenditure 
Per Family 
[C/B] 


1989 


76 


2,460 


32.4 


$14,820,000 


$195,000 


$6,024 


1990 


122 


6,596 


54.1 


$24,201,000 


$198,369 


$3,669 


1991 


239 


14,900 


62.3 


$49,770,000 


$208,243 


$3,340 


1992 


340 


20,800 


61.2 


$70,000,000 


$205,882 


$3,365 


1993 


490 


29,400 


58.0 


$89,123,000 


$181,884 


$3,031 


1994 


513 


27,200 


53.0 


$91,373,000 


$178,115 


$3,359 


1995 


576 


31,500 


54.7 


$102,024,000 


$177,125 


$3,239 


1996 


637 


34,400 


54.0 


$101,997,000 


$160,121 


$2,965 


1997 


655 


35,370 


54.0 


$101,997,000 


$155,721 


$2,884 


1998 


732 


39,528 


54.0 


$124,000,000 


$169,399 


$3,137 



Sources: ED program data for funding and number of local grants; evaluation contractors (Abt Associates 

Inc., Pelavin Associates, Inc., and Fu Associates, Ltd.) for participants. 

Note: 1 . Federal Even Start expenditures include funds for technical assistance and evaluation, and state 

administrative funds. Subtracting these funds would not change the conclusions drawn in this 
report. 

2. The federal Even Start cost per project and cost per family are calculated by using federal-level 
data on the total program expenditures and the total number of projects funded, combined with 
project-level data on the total number of families served. 
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Exhibit 2: Model of Even Start's Hypothesized Effects 
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Exhibit 3 

Educational Gains for Even Start Children, Adults, Parenting 



GAIN (Standard Deviation Units) 

0 0.2 0.4 0.6 0.8 1 




Note: Gains as measured over a 1-year period. Data taken from the first and second national evaluations. 
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Exhibit 4: Even Start Family Literacy Program Performance Plan: 
Objectives and Indicators 



Objective 1. The literacy of participating families will improve. 

1.1 Adult literacy achievement. Increasing percentages of Even Start adults will achieve significant 
learning gains on measures of math and reading skills. 

1 .2 Adult educational attainment. Increasing percentages of adult secondary education Even Start 
participants will obtain their high school diploma or equivalent. 

1.3 Children’s school readiness and success. Increasing percentages of Even Start children will 
attain significant gains on measures of language development and reading readiness. 

1.4 Parenting skills. Increasing percentages of parents will show significant improvement on 
measures of parenting skills, home environment, and expectations for their children. 

Objective 2. Even Start projects will reach their target population of families that are most in need of 

services. 

2.1 Recruitment of most in need. The projects will continue to recruit low-income, disadvantaged 
families with low literacy levels. 

Objective 3. Local Even Start projects will provide comprehensive instructional and support services of 

high quality to all families in a cost-effective manner. 

3.1 Service hours. Increasing percentages of projects will offer at least 60 hours of adult education 
per month, at least 20 hours of parenting education per month, and at least 65 hours of early 
childhood education per month. 

3.2 Participation, retention and continuity. Projects will increasingly improve retention and 
continuity of services. 
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